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Studies of transfer of training may be used to 
determine whether simulator training improves pilot performance in an 
aircraft* Soie approaches to determining simulator training 
ef f ectivenesSy such as surveys of pilot and instructor opinions^ are 
not considered particularly reliable. Several other approaches have 
also been suggested. One xactor influencing simulator training 
effectiveness is simulator design, including fidelity of simulation 
and design for training* Pidelity refers to the extent to which 
features of the aircraft and its environment duplicate their real 
world counterparts. Studies on the influence of such features as 
visual fidelity, action fidelity, and handling characteristics are 
somewhat inconclusive. Design for training refers to the inclusion of 
features which facilitate training but which may bear no particular 
reseoblance to features of the aircraft and environment being 
simalated* The .sequencing of simulator and aircraft training and 
training program content are also influences. Past experience, 
aptitudes, skills, attitudes, and expectations of trainees and 
instructors represent potential influences on effectiveness of 
simulator training, studies show that expectations for simulator 
training are usually fulfilled* (LMS) 
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SOME FACTORS INFLUENCING TRANSFER OF 
SIMULATOR TRAINING 

Paul W, Caro 
Human Resources Research OrganiEatinn 

INTRODUCTION 

The use of simulators and other training devices is incrtwihg rapidly. One assumes 
that this phenomenon is intended to enhance the effectiveness and /or the efficiency of 
the training programs in which such devices are used, but the seriousness with which 
simulator makers and users pursue this intent might well he questioned from both the 
practical and scientific points of view. Following a study of the use of aircmft simulators 
in selected U.S. military and civilian pilot training programs, the Comptroller General of 
the LJnited States recently issued a report to the Confess which was critical of the 
extent to which simulators are being used in the military training programs studied (1), 
The report suggested that present knowledge of simulator design and employment is 
sufficient to support mucii more extensive use of simulators than v/as tyt^ically found to 
he the case. The report cites certain pilot training agencies which seem to employ that 
knowledge more effectively than do others, even though the knowledge is available to all. 

The Comptroller General report was not intended as a scientific study of psy= 
chological factors and their influences on simulator training effectiveness. Instead, it 
identified factors related to the management of simulator training and attitudes toward 
such training which tend to impede more extensive use of simulators. Factors identified 
include regulations emphaslEing aircraft rather than simulator training, inadequate instruc- 
tor training, failure to use simulator capabilities fully, and poor iimulator maintenance. 
The influence of such management factors upon trainer use has been documented in the 
rese^ch literature (e,g., 2,3). Other factors which have been identified as impediments to 
effective simulator use include the design of the simulators themselves and of the training 
pro-ams in which the simulators are employed (e.g., 4,5), Still other factors have been 
reported which influence the acceptance of simulators by those who must use them and 
thereby indirectly impede their more extensive use (e,g,, 6), 

Ctorly, the intent of simulator users is to achieve effective training rather than 
merely to use training equipment extensively. It is possible, however, to use simulators 
extensively, while at the same time to use them ineffectively. In one study (7), for 
example, it was found that the extensive use of a particular device added cost, but no 
training value to an already expensive pilot training program. Another study demon- 
strated that the training value of a device could be increased substantially without 
increasing the amount of device training time involved when the manner in which it was 
used was changed (S). 

While there is a ff^mi deal of scientific and training literature in existence dealing 
with simulator trainiEng, some of the more significant factors influencing transfer of 
simulator training have not received the systematic attention they warrant. The purpose 
of this paper is to call attention to certain of the factors observed over the last decade to 
be of significance in oui simulator research and that of our professional colleagues 
elsewhere. The emphasis will be upon increasing the effectiveness of simulator training as 
opposed merely to increiising simulBfor use. While simulator effectiveness and use 



obviously are not independent (a simulator cannot be used effectively when it is used 
little or not at all), they are not always coincident. Some simulator designers and users 
appear unaware of factors which, if properly treated or managed, would markedly 
enhani*e the value and efficiency oj simulator training programs. The unfortunate conse- 
quence i^f their lack of awareness is that simulator training effectiveness suffers: skills 
which may be critical to safe operation uf an aircraft may not be developed; aircraft may 
be required unnec^sarily for training when they are needed for other purposes; training 
costs become excessive; or simulator training, although effective, is conducted in an 
inefficient manner. 



MEASURING SIMULATOR TRAINING EFFECTIVENESS 

In order to identify factors which influence simulator^ training effectiveness, it is 
necessary to measurt those influences. Since simulator training effectiveness^ is not 
indcfH'nde^t of tnc extent to which the device is used, one indication of whether a factor 
can r)e iniiuentiul is to determine how much a simulator training pro-am embodying that 
factor is used, e.g., the elapsed time that the device is in use, or tne number of "asRs 
pracLiced in it. While simulator training cannot be effective if these values equal zero, 
hiyh inriices of im^ do not indicate effective simulator training. Even improved pilot 
performance in thf* simulator resulting from its use does not assure improved performance 
in the aircraft. Therefore, measures must be used which reflect improvements in pilot 
performance in the aircraft, not how proficient the pilot became at flying the simulator 
or how much training he received in it. 

The Transfer Model 

The transfer of training model can be employed to determine whether simulator 
training has improved subsequent pilot performance in an aircraft. Transfer of training is 
a phenomt^non which occurs whenever the existence of a previotisly leamed behavior or 
skill has an tnnuence upon the acquisition, performance, or reieaming of a second 
behavior or skill. Thus, if a behavior learned in a device has an influence upon the 
subsequent acquisition, performance or reieaming of behavior in an airplane, transfer is 
siud to have occunred* 

A simple experimental design bastHj upon the transfer of training model involves two 
^oups of trainees: an experimental group which receives simulator training prior to 
further training or performance testing in the aircraft; and a control group which receives 
all Its training in the aircraft. This design permits measured differences in performance in 
the aircraft between the experimental and control groups to be attributed to the 
innuenV'e of training received in the simulator by the experimental poup. The groups 
must \ye equated, of course, in terms of relevant prior training and experiences, and a 
means for measuring in-flight performance must be available, 

**5lany writers di^stinyui^ih bc'twt'L'n gimulat<?rs (t',|*., , .a hiKh dey^tvo of relcvanCG to operational 
^uipmi»nt. , and training fi«?vices (t^g., .any piece of apparatus which is used for training,,,*'), 
Tti** pr*^nt wfitef will tre^at thust €?xarripi«?.s of tFaining equipment as mombers of a single class. To 
(juote Gagne (9, p. 96), the source also of the above quotations, "What distinguishes a training device 
(frfim other training equipment) in not its appearance or construction, but rather how and for what 
purpmes it is used/* The term simulator m used here to identify ground based training equipment used 
for the purpose of training pilots to fly aircraft. The question of how it is used will be addreSi^d in the 
pap*^r. 

^, . , or, us more properly stated by Jeantheau ( 1 0), the effectiveness of training with the 
Simulator. , . 
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Other Approaches to Determining Simulator Training Effectiveness 

Dt'termininii tht* efft»ctivi?nf*;s of Himulator training can ht* a custly, time-consuming 
task if an in^fli^ht perfurmance measurement system is not readily available. Some 
analyriL?4 have attempteti to eonduct simulator ovaluationH by asking |)ilots, instructors, 
iramina sperialist^^ and even ntudenti^ their opinions concerning simulator effectiveness, 

the prol>ahle impact upon student performance of training in the simulator. I have 
ohrierved instances in which surveys of pilots' and instructors* opinions yielded results 
I both pro and con) unrelated to data subsequently obtained in transfer studies involving 
the devices m question. Meister. Sullivan, Thompson & Finley (11) found that such an 
approach yieldetl different estimates of the effectiveness of a particular devic^e where 
tiifferent instructors made the judgments involved. Instructor evaluations of simulator 
tfffectiveness are unreliable, probably due to attitude factors such as those discussed by 
Mackie, et. ah {6), as well as because of the inherent unreliability of judgments, and 
should not be used where much is at stake. Simulator effectiveness is a matter of trainee 
m^night performance, not of instructor, pilot, or trainee opinions about the device and its 
pro bub le usefulness. 

Another simulator evaluation technique, backward or inverse transfer of training, has 
bf-en deserilied by Adams and NlcAbee (12). In a backward transfer study, a pilot who 
has already demonstrated mastery of in-flight training objectives is ^^transferred'' to the 
simulator where he is required to perform tasks corresponding to those he has mastered 
in ihe aircraft. If he can perform such tasks at criterion levels in the simulator without 
practice there, backward transfer is said to have occurred, and this fact is token as 
evidence that transfer in the simulator=to<levicc sequence, although of unknown quantity, 
will be positive. The backward tmnsfer design should be used with caution for at least 
thrive reasons: (a) the results assume (often incorrectly) that a suitable training program 
exists for the simulator; (b ) experienced pilots already proficient at operational tasks 
often have generalized skills not possessed by recent graduates and may be able to 
transfer to the device because of such general skills rather than skills needed to pilot a 
particukir aircraft or perform a particular mission; and (c) the simulator may be suitably 
designed for the performance and evaluation of a particular set of behaviors but may lack 
the cues essential to the development of those behaviors. While backward transfer data 
should nut be the sole justification for simulator procurement, one would be hesitant to 
use a simulator which could not be operated by competent pilots, 

Jeantheau (10) has described four approaches to simulator evaluation: (a) conduct of 
a transfer experiment as is described above; (b) analytic study of the device and its 
program of instruction; (c) use of the device without experimental controls which would 
permit measurement of its effectiveness; and (d) comparison of various ways of using the 
device. These latter three approaches are appropriate to assuring that a simulator is used 
correctly, but they do not yield data which demonstrate that pilots who are trained in 
the .simulator are more competent than pilots who are not so trained. Sometimes such 
approaches must be relied upon for administrative reasons or because experimental 
controls involve unacceptable risks (e.g,, the effectiveness of lunar landing simulators 
could not lie determined in a study involving a no-simulator4raining control group). 

The fact that simulator training research cannot always involve transfer to an aircraft 
is not necessarily all bad. Many research issues, including issues related to efficient 
methods of simulator training, can be investigated in simulators and laboratories to 
considerable advantage. A higher degree of control can be exercised over independent 
variables in such studies, and the cost of aircraft operations c^n be avoided. The chief 
disadvantage lies in the resulting uncertainty concerning the effectiveness of that training 
in the operational situation. Recent efforts of Matheny (13) on perceptual equivalence 
may offer a means of testing simulator effectiveness without costly transfer studies. 



A number of analytic models have been suKested for predicting simulator effective- 
ness (14, 15, 16, 17), but none of them takes into account all of'the factors believed 
relevant. They are useful in designing simulators and training prop-ams, but they do not 
measure simulator effectiveness and wtimates based upon them are subject to 
unknown eiror. 

Indices of Simulator EffectlveriBs 

Various formulae have been suggested to express simulator effectiven^ as a single 
value. Gagne, Foster & Crowley (18) proposed nine such formulae, corresponding to 
different operations involved in deriving the raw data, e.g.. counting trials, measuring 
time, or counting errors. Six formulae have been suggested by Murdock (19) and five by 
Hammerton (20), each of which deals with specific experimental design and data 
problems. An index which takes into account the amount of effort involved in device 
ttaimng as weU as subsequent in-flight performance, the Transfer Effectiveness Ratio 
(TER), has been proposed by Povenmire & Roscoe (21). A variation of the TER takes 
mto account the relative cost of simulator vs. aircraft training (22). While aU such indices 
provide a means for comparing simulator effectiveneK, they contribute Uttle to our 
understandmg of the training value of simulators. The raw data themselves, a description 
of circumstances under which the measures were made, and Identiflcation of the 
dependent variables are more useful In understanding factors influencing simulator train- 
mg. If a single mdex is necessary, it might better be one which reflects the cost savings 
resulting from the use of the device in a specified way in a particular training propam. 

FACTORS INFLUENCING SIMULATOR TRAINING EFFECTIVENESS 

A recent summary of simu]p»or ttaining studies (3) indicated that simulator trmning 
effectiveness has mcreased markedly since World War II. Some of the increase can be 
attnbuted to advances in engineering and instructional design technologies. Simulation 
engineers now have the technology available to build simulators which more nearly satisfy 
Thomdike-s common elements design hypothrais, and instructional system designers have 
learned how to zero in on tasks to be trained. But much of the increase has come about 
as a result of research and experience vrith simulatore in operational training settings. 

In spite of the large number of simulator effectivenras studies which have been 
completed, there remains mucli to be learned about training with these devices Factors 
have been noramated as influences upon simulator training effectiveness, but many of 
those nommations have been based on inference rather than experimental evidence In 
their review of simulator research, Muckler. Ny^d. O'Kelley, and Williams (23) noted 
that many studies compound the influence of several potential influents such as training 
program content, mstructional technique, and instructor qualification, into a single 
independent variable so that the transfer effects can be attributed only to the unique 
combmation of those influences. Even in the few experimental investigations which 
isolate assumed influences, the results must be interpreted cautiously because they 
address unique teammg requirements and have not been replicated. 

The methodological problems involved in Identifying factors which influence 
simulator training effectiven^ cannot easily be overcome. Suspected factors can seldom 
be examined in isotation. It is difficult, for example, to determine experimentally the 
relative value of a remote instructor station vs. an on-board or in-the-cockpit station even 
if a suitably designed simulator were available for the research, because to use each 
station to its best advantage woukx necessitate having two methods of teaining- one 
optimized for remote instruction; the other optimized for on-board instruction The 
expenment would thus compare Instructor itatlon-training prop-am combinations not a 



simulatur desian ft*aturt> in isolation from other factors. The Gaining program factor 
cunnui be ht^kl constant. It would he inappropriate to compare two simulator desmns 
usiriL? a prfj^ram optimized fur only one, or for neithor, 

" rlw prohlum of generalizahlc results is not limited to studies involvinp operational 
simulators. Even using equipment designed and dedicated to research, problems arise. For 
example, lu pursue the illustration of instructor station location described above, 
simulator hardware innexibility makes it difficult to conduct the necessary research 
leadma to the design of the optimum remote instructor station for expenmental com^ 
parison with the optimallv designed on4)oard station. Additionaliy , the on^board station 
cU^smn which is optimum fur a single seat, high performance attack aircraft simulator with 
a visual display may beur little resemblance to the optimised on^board station for 
uiuUT^TOduate insLrument training in a side^iy^side seating holicopter simulator. 

Definitive data do not exist which will permit the quantification of the mfluence of 
all factors believed to innuence simulator training offectiveness. In fact, the mere 
identification of most such factors rests upon inference, conjecture, and untesteu 
hypotheses. The absence of hard data obviously cannot justify suspected factors being 
Ignored, nowever. Where inferences can be made and supported by consensus, factors 
believed to inOuence simulator training must be taken into account by those responsible 
for simulator design and use unless evidence can be assembled to refute those inferences. 
It is the responsibility of the research community to undertake the systematic investiga- 
tion of such factors. < 

The following discussion is intended to ^^11 attention to selected factors which 
influence simulator training effectiveness. It would not be fruitful to attempt to cite all 
the re^. r^'hers who have contributed to the identification of such factors^those who 
have contributed to the literature on physical vs. psychological variables in simulator 
design for example, are legion, as are those who have remarked upon the importance of 
how the simulator is used. Review articles touching on the subject include references 
no. 2, 3, 23, 24 and 25, Except where specific reference is made to a particular report, 
the present Witer will acknowledge responsibility for the inferences set forth herein, as 
well as for the selection of factors to be discussed. 

Simulator Design 

There are two areas of interest with respect to the influences of simulator design 
upon transfer of training: fidelity of simulation and design for training. Fidelity refers to 
whether features of the airiTaft and its environment are included in the simulator's 
design and the extent to which features which Bie included represeqit or duplicate their 
real world counterparts. Design for training refers to the inclusion in simulator design of 
features or configurations which facilitate training but which may bear no particular 
resemblance to features of the aircraft and environment being simulated. 

Fidelity of simulation is often equated with physical correspondence between the 
device and its real world counterpart. In their discussion of simulator design considera^ 
fions however, Smode and Hall (26) emphasize instructional strategies and capabihties 
and suggest that fidelity has meaning in terms of the process and the realism necessary to 
prnmot^ learning. Design characteristics, they assert, should be defined in terms of 
assurinc. transfer of training. In other words, fidelity of simulation is a matter of the 
relevance of the simulation to the training objectives, not solely a matter of physical 
correspondence. This concept of fidelity accounts for the effectiveness of so-called low 
ndelity devic es as well as simulators that faithfully reproduce much of the aircraft. 

Design for Training. The Smode and Hall concept of fidelity i_of particular interest 
with respect to si^mulatSr features not modeled after the aircraft. These features, which 
are concerned Firimarily with application of principles of learning to the trammg process. 
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.m.lu(h. freoze. adaptive training, prompting and cueing, performance recording and 
Playluu.k. performance measurement, and various instructor station displays and controls 
I 1.^ U^.neral y held that such features improve the conditions under which learning takes 
Piac*^ and thon-hy facilitate the attainment of training objectives. Therefore they are 
ta.Lors to he considered in judging the fidelity of a device so far as training is concerned 
It IS u.MiGral practice to adopt innovative simulator design features such as those 
mont.onod ar,uve on the basis of their apparent utiUty without subjecting them to 
espenm,.nta scrutiny. For example, the widely used simulator freeze feature was 
imp..ment,Hl because it was seen as an aid to attaining training objectives and to 
impk-mt.nt.ni; kumirng concepts during the instructional process. Similarly other design 
■KMStons ar. made because the training objectives and planned concepts of simulator 
ernploymrnt lead to the conclusion that a particular design is appropriate in preference to 
others For exampk.. m the design of U.S. Army simulators for the Vertol CH=47 and the 
.kL , hel.copu.rs. the instructor stations were located virtually inside the cockpits of 
ihtse devnces, and certain instructor displays were positioned so that they could be 
viewed by both the instructor and the trainees in order to facilitate instructor-trainee 
interac ,ons dunnc key training activities. The training effectiveness of these features 
[,rubably will never be determined in a transfer experiment for the methodology reasons 
discussed above. Analytically, they are believed to represent effective simulatS designs 
w,in rt:sPf« to the .Vrmy's training program and the training objectives to be addressed 
Njsua^^idehty. Generally, tasks which cannot be duplicated or even approximated 
m a device cannot be learned there for subsequent transfer to the aircraft. Therefore a 
.simulator in which more tasks characterizing flying can be performed has ffeaier 
potential training effectiveness than one in which fewer such tasks can be performed For 
e.^cample, a simulator which does not include an extraK:ockpit visual display would seem 
to have less effectiveness potential with respect to training tasks requiring visual 
references than a simulator with such a display. h b ^u*" 

There have been a number of studies in which transfer from a simulator with a 
visua^l display has been demonstrated. The scenes presented by some of these displays are 

r^auLrf f 'T'^ ^" '"""^P^^' in aircraft time 

required to perform visual reference maneuvers were demonstrated' in a study by 
Flexman. .Matheny & Brown (27) using a simulator with a visual display consisting of a 
line drawing on a blackboard placed in front of the cockpit and tilted by an instru^r to 
cliange perspective as the device was maneuvered with respect to simulated ground 
references. The effectiveness of other simple displays consisting of stylized grids and Unes 
has _been demonstrated^ in backward transfer situations during studies of contact analop 
displays developed for helicopters (28), Displacement of scene elements consisting only of 
dots and lines was found by Thielges and Matheny (29) to provide sufficient information 
for the performance of aircraft control tasks, although their study was not based on a 
transfer model. 

These studies indicated that tasks involving aircraft control in relation to extra- 
cockpit visual information can be practiced effectively In simulators with very simple 
visual scene displays. The displays consisted of no more than points, lines and geometric 
^ttems arranged in accord with a set of mathe.natical relationships described by 
C^ibson (30). Several manufacturers currently are taking advantage of the utUity of these 
simple scene content design requifements by marketing displays which represent niaht 
scenes as patterned points of light on a black field, and their displays are being used ^th 
apparent success in commercial airlines' simulator training pro-ams 

While the effectiveness of such simple visual displays has been demonstrated to the 
extent descnbed above, it is also noted that simulators without a visual display can be 
effective m the training of visual reference flight tasks. In a study involving a helicopter 
•simulator without a display or any other representation of outside visual cues except the 



ERIC 



aircmft's navigation and attitude instrument, and without any attention during simulator 
training to oxtraHjockpit visual cues per se, students trained to fly instrument flight 
missions m the device qualified in the aircraft under visual conditions more rapidly than 
did students not receiving the prior device training (31 ). In a simUar study using a fixed 
wing simulator without a visuaJ display, a saving in visual flight time required to complete 
a tTansition course of approximately 50% of the scheduled coarse length was 
obtainod (8), An unreported study by the ILS. Air Force involving cognitive training in a 
simulator with no visual display demonstrated transfer to visual flying maneuvers such as 
traffic patterns (32). 

While it is not indicated by these studies' results that visual displays have no trammg 
value, it appears that many behaviors reciuired as responses to extta-cockpit visual stimuli 
in the aircraft can be practiced-or at least approximated=in response to stimuli in a 
simulator without an outside display. Further, cockpit instruments provide information 
about and an analog display of the visual world outeide the cockpit, so a pilot flying 
instruments is responding to stimuJi analogous to those available to the pilot flying 
visually. At least some of the simula^nr effectiveness attributed to the simpler visual 
displays probably would occur without the presence of such a display at alL 

During the current review, no studies were found which unequivocably estabUshed 
the effectiveness of any extraH^ockpit visual display. While transfer studies mvolving visual 
displays were found/ only one, an exploratory study judged inconclusive by its 
authors (33), included a control ^oup in which students were traimd in the simulator 
without using its display. Commerical airlines have reduced aircraft training time fol^ 
lowing the addition of a visual display to an existing simulator, but some if not all of the 
reduction resulted from a priori judgments by government agencies and the airlines 
themselves concerning increased simulator tjaining effectiveness. In no cases have there 
been reports of efforts to design training pro-ams which would seek the same flight 
training mvings using simulators without visual displays that presumably have l>e€n 
achieved using simulators with such displays. 

The lack of evidence of visual display training effectiveness cannot be taken as 
evidence of their lack of effectiveness. There is a consensus that they are effective, and 
daU to contest that consensus do not exist. Logically, it would appear that an extra^ 
cockpit visual display is an effective way to present visual Information used in some 
operational tasks^uch as landing on a carrier, taxiing, refueling, delivering certain kinds 
of weapons, and air-to-air combat. In some instances, it may be the only effective way. 
In others, it may he effective, but inefficient, particularly when cost is taken 
into consideration. 

Motion Fidelity. Not much more is knowTi about the influence of motion upon 
simulaiSr tminini^ effectiveness than about visual displays. Although motion simulation 
has represented a signincant portion of the cost of simulator procurement and operation 
for a number of years, the investigation of the influence of motion upon simulator 
training effectiveness has been largely ignored. The first significant study involving 
simulator motion in the transfer of pilot training was reported in 1975 by Jacobs and 
Roscoe(34), 

The results of the Jacobs and Roscoe study provide evidence that transfer may not 
benefit from the presence of normal washout cockpit motion. In that study, training 
received in a two^axis normal washout motion condition, compared with training in the 
same device without motion, resulted in non^significant differences in amount of transfer 
to the aircraft for these two conditions. There was however, significant positive transfer 
for both motion and no-motion conditions, SimUar results have been obtainod in an 
unpublished U.S, Air Force undergraduate pilot training study involving a more 
sophisticated six^axis motion system (32), 
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Ihf .Jacobs and Roscoi? finding that, at least \vith bin'innirm trainees, the presencp of 

mow may not increase simulator training effectiveness must be treated with eaiUion 
unul invesiitfated further, since there are other studies suggesting that, at least under 
Mmw circumstances, motion may be desirubie even if not essential. For example 
l-eddersnn (35) reported a slight advantage in favor of a motion-simulator trained group 
uvfr a no-motiun group during brief transfer trials hovering a helicopter, "wore 
importantly, perhaps, the motion group in his study reached asymptotic performance in 
the simulator more rapidly, suggesting that simulators with motion may provide more 
etfic.t.nt training, even if not more effective training. NASA researchers (36) have found 



tha' tiie correlation Ijetween pUot performance in an aircraft and in a simulator in,_.trases 
vvi fi ihv addition of simulator motion cues where such cues help the pilot in coping with 
a .uuhly damped or unstable vehicle or a sluggish control svstem. or under some 
vin utnstances, where the control system is too sensitive. Where the aircraft is .-asy to fly 
however, a.s is the case with the aircraft used in the Jacobs and Roscoe study (i>iper 
Cherokee) and m the Air Force study (T-37), motion has no effect. In another NASA 
study (3/) of the effects of simulator motion on pilots' performance of a tracking task 
the results from a moving flight simulator resembled the results from flight much more 
thtm did tho.se from a motionless simulator. Huddleston and Rnlfe (38) reported that 
iLsint' simulators without motion, experienced pilots are often .able to achieve acceptable 
levels uf ptTformance, but th-ir pattems of control respon.se show that their performance 
L< acnieved using a strategy dilierent from that used in a dynamic training environment 
bmce control strategies may be important during in-flight emergency maneuvers where 
transter oi training re.search is not feasible, it would appear inadvisable to eliminate 
mntion r mm all .simulators until further investigation shows the generality of the Jacobs 
anci Ftnscoe findings. At the present time, we cannot be certain of the role of motion in 
simulator training effectiveness and efficiency. 

HandUng Characteristics. Simulators built with the technology available two decades 
ago tended to have handling characteristics which were sometimta quite unlike those of 
an au-craft. and their effectiveness was limited largely by the fact that pUots resisted 
traming m them or wouJd use them only as procedures trainers (5). There were-tind still 
are=strong pilot opinions that a simulator had to "feel" like an aircraft if it were to be 
effective. Transfer studies of individual aircraft control parameters, such as a study of the 
correspondence in- stick pressures between a device and a training aircraft (39) faUed to 
lend support to the pilots' opinions. Where the correspondence between the device and 
the au-craft is gross, however, as v. as found in one device in which forward pressure on 
the wheel resulted in a climb configuration (40), simulator effectiveness undoubtedly will 
suffer. Thus, dthough in the extreme case simulator response characteristics unlike those 
of the aircraft can produce negative transfer of training, there is little evidence that the 
simulator must precisely duplicate the feel of the aircraft in order to be effective. It Ls 
possible, however, that even minor dissimilarities in feel or res- nse could lead to the 
same kinds of potential problems found in simulators withe motion, i.e., lower 
correlation between simulator and flight performance, particularly ..here the more diffi- 
cult to fly aircraft are concerned. 

Our understanding of simulator design features in relation to simulator training 
effectiveness is quite limited. It is clear that designing a simulator is not entirely a matter 
of duplicating an aircraft. The physical correspondence between the simulator and the 
aircraft is probably more related to cost, as Miller (41) indicated almost two decades ago 
than to training effectiveness. If the degree of correspondence between the device and the 
aircraft is relevant to the objectives of the intended training, training in the simulator can 
in- made effective— whether it is or not is a matter related to other factors. 
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Training Programs 

FrfeHjuent note has bwn taken of the influence upon training effectiveni-ss of the 
manner in which a simulator is used. Yet, the literature is full of reports of situations in 
which the importance of training pro^ani design and execution seemed to be 
ikmorecl (e.g., 5), Although there is an increasing emphasL^ upon effective use of devices, 
Juirent instances can be cited of training propams in which simulators are misused or are 
used mefficiently. Even in simulator effectiveness research, participating instructors often 
are permitted to conduct training in various non-stond^dized ways. 

Id li^t ail training pro-am design and execution variables which potentially 
influence simulator ttaining effectiveness would be an almost intarminable task. Any of 
the numerous textbooks on human learning will provide a source for identification of 
vanahles which Influence learning and performance, e.g., schedules of reinforcement, 
meaninicfulness and difficulty of material to be learned, size of learmng blocks and 
knowledge of result, Flexman, et al,, (27) have shown how such variables can be 
*^ni ployed to increase simulator and flight training effectiveness. 

The sequencing of simulator and aircraft training has been suggested m a factor 
which could influunce the effectiveness of simulator ttBuning. Smode, et al, (2) concluded 
that the evidence concerning whether sequencing is influential was inconclusive, Meister, 
et ai ill) presenteti data which suggest that ^witching from the aircraft to the simulator 
reduces performance in the snnulator on the following sessions, resulting in a trammg 
inefficiency. WhUe there may be some interactive effects between the sequence, the 
manner m which the device is used, and the design of the device which could influence 
effectiveness, it would appear quite Ukely that training in the aircraft before the full 
henefit of the simuiator has been realized with respect to a particular task would tend to 
reduce the overall efficiency of the simulator^evice Gaining propam. In an unpublished 
instance which illustrates this view, a fifty ttaining hour propam in which the simulator 
was used prior to training in the aircraft became a sixty training hour propam when the 
sequence was changed to mix simulator and aircraft training, althou^ other changes were 
introduced concuirently which could have contributed to the resulting inefficiency. 

Training pro^m content is an obvious influence upon Emulator training 
effectiveness. A dynamic flight simulator used only as a procedures trainer, for example, 
is not being used effectively. It Is also believed that simLaator training praented in the 
context of simulated mission activities, as opposed to abstmct Gaining exercises, tends to 
he more effective, and the literature on learning and forgetting suggests that behavior 
learned within such a meaningful context will be less quickly forgotten (42), 

There are a number of other training pro-am factors which influence smiulator 
training efficiency and thus would lead to a higher TER value, althou^ not to increased 
effectiveness per se. Thes^ include the amount of simulator training, the sequence m 
which instruction is conducted in the simulator, the use of individual oppo^ to 
^oup) pacing, training to specified criterion levels (as opposed to toimng for fixed time 
periods) and the extent to which simulator training includes tasks which be learned 
more efficienUy in the aircraft. Smode, et aL, (2) pointed out a decade ago that Uttle 
was known about how to manipulate such factors to best advantage. That observation is 
still valid. 

Personnel 

Simulator training involves trainees and instructors. Both categories of personnc^l 
represent potential influences upon effectiven^s. The most obviously relevant considera- 
tions with respect to both are their qudiflcations and prior experience, but occasionaUy 
other variables are suggested. For example, Meister, et al, (11), found a difference m the 
effectiveness of one simulator training program for student and operational pilots vs. 



reservists. The differenee could also be attributed to considerations such as fatigue and 
^ess, factors which prohably account for many unexpected findings In transfer studies 
The present paper wUl discuss only the more obvious personnel factors. 

Trainees. All Investigations of human learning are subject to the influences of 
tajk-related aptitudes of the learners. Aptitudes are defined in terms of learning 
efficiency, and high aptitude students learn a given task more rapidly or to a greater 
degree than do low aptitude studente. Where the training program involves fixed amounts 
of simulator training time, high aptitude students learn more tasks to transfer to the 
aircraft; where traini.ig is to fixed pfirformanoe levels and training time varies high and 
low aptitude studente achieve about equally, but high aptitude students requLre less 
Gaining time m the simulator. A measure of simulator training efficiency such as the 
rtR wUl yield a higher value for high aptitude students, but this does not indicate that 
thu simulator training prop^m is more effective with such students. It is robably equally 
effective with both groups of students, but training time in the device wUl be shorter for 
one than for the other. Thus, whUe high aptitude students learn more efficiently 
aptitude per se is not beUeved to be an influence upon simulator training effectiveness. ' 

The influence upon simulator training effectiveness of level of trainee skill or 
amount of prior flight experience is frequently questioned. Many military pUots and 
managers acknowledge that simulators provide appropriate training for the airlines where 
the trainees are highly experienced, but insist that the deivces cannot be relied upon as 
extensively to train less e.xperienced military pilots. The skills possessed by these two 
ffoups of trainees do differ, qualitatively as well as quantitatively and the tasks for 
which they undergo braining are not identical. Therefore, the training they receive should 
not be Identical if it is optimally designed to meet their respective training needs, and the 
characteristics of the simulators involved in their training should vary as well. It does not 
follow, however, that simulator training can be appropriately designed and conducted for 
one experience level ttainee but not for another. In fact, the experimental evidence does 
not support the contention that simulator training effectiveness is influenced by level of 
trainee expenence in isolation from other factors. After reviewing a large number of 
transfer of training studies, Micheli (3) concluded that flight training devices are effective 
for both neophyte pUot trainees and airline pilots. 

Instructors. After reviewing the literature on the flight instructor, Smode et al (2) 
concluded that experienced pilots do not make better in-flight instructors than 
inexperienced pUots. The same conclusion can be extrapolated to simulator instructors 
While the evidence is skimpy, it appears that even personnel with no flight experience can 
be trained to be effective simulator instructors. For example, in a simulator training 
study comparing an instructor with several thousand hours military instructor-pilot 
expenence, a recent flight training program graduate, and a non-rated individual with a 
few hours dual instruction but no other aeronautical experience, no significant differences 
were found in the in-flight performance of their students (43). 

There is some evidence that not all simulator instructors are equally prepared for 
their job. HaU, et al. (5) surveyed a number of military training programs and found that 
non-rated enUsted instructors were ill prepared as compared with pilots, particularly with 
respect to relevant knowledge of the aircraft. They also noted that pilots were simUarly 
lU prepared with respect to knowledge of the capabilities and limitations of the simu- 
tators. Since no transfer data were reported, it cannot be determined whether this factor 
had an influence upon subsequent in-flight performance in favor of either type 
of instructor. 

Muckler, et al, (23) observed that in some cases a simulator instructor must provide 
supplementary information about the in-flight task which might not be available to a 
non-rated mstructor, thus presumably tipping the scale in favor of pilots as simulator 
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instructors, Muckler, et al., also noted that instructor ability and fidelity of simulation 
are related in such fashion that as fidelity incre^es, the neces^ry level of instructor 
ability may decrease, and, conversely, as fidelity decreases, instructor ability must 
increase. This relationship would tend to place the more able instructor in the lower 
fidelity simulator where a peater amount of supplementaiy information might be 
required. It has benn my observation that just the opposite situation often obt^ns. The 
more eK^rienced pilots instruct in high fidelity simulators, while less ex^rienced and 
non rate 1 personnel instruct in older, lower fidelity devices. 

Another consideration is whether there should be one instructor or two in a 
simulator training propmni. That is, is simulator bfainirif effectiveness influenced by 
whether the simulator instructor is also the in-flight instractor? While this variable has 
not been isolated for study, there appears to be an increase in effectiven^s when a single 
instructor Is responsible for both simulator and aircraft training, and it has become a 
standard feature of the simulator training propams developed by my organization,^ One 
apparent benefit is that the instruction given in the simulator is more compatible with 
that given in the aircraft when only one instructor is involved, thus reducing any 
potential negative transfer attributeble to instructor-peculiar performance requirements. 

It often has been assumed that the instructor is an important factor influencing 
ta^aining effectiveness, and such may well be the case. If so, the influence must be 
attributable to the manner in which the instructor functions, i.e., to non-standardization 
in his administration of the training pro-am. There is insufficient evidence available at 
this time to attribute the assumed influence to instructor experience or qu^ification per 
se-^^suming he has undergone an instructor training propam appropriate to the instruc- 
tional task at hand. 

Attitudes 

While the influence of simulator design upon simulator tt'aining effectiveness may 
not always be clear, simulator design has an im^ct upon instructors and trainees, 
reflected in their attitudes, which in turn has a large influence upon simulator training 
effectiveness. Flexman described this impact as follows (quoted in 23, p. 69): "Fidelity of 
simulation can operate as a motivational variable. If the simulator looks^ acts, feels and 
sounds like the airplarie, then the trainee is more likely to be convinced that practice in 
the device will be beneficial to him/* In circular fashion, attitudes also influence 
simulator design, WiUiges, et al. (25) noted this phenomenon when they stated that 
decisions to include complex and expensive motion systems in simulators bib inveuiably 
determined by pilots* attitudes. It has been ray observation that fidelity of simulation has 
a greater impact upon the attitude of the simulator instructor, particul^ly if he is a pUot, 
than it has upon the toainee, and, in turn, instructor attitudes concerning simulator 
fining can determine teainee attitudes. 

The most direct effect of tarainee and instructor attitude upon simulator teaining 
effectiveness is probably upon their willingnei^ to engage in simulator training in the first 
place. That is, devices which are viewed favorably seem to he used more than those 
which are viewed less favorably. If the addition of a motion system or visufd display to a 
simulator will result in favorable tramee and instructor attitudes tow^d simulator training 
and hence peater utilization of the device, it is possible that more effective simulator 
training will result from the peater utilization, even though the motion and visual per se 
may contribute nothing directly to transfer* 



^The single instrycior concept hits been used elsewhere at least as early as the late 1940s (44), 



It would be^ mistake to attribute all favorable attitudes toward simulator training 
to high fidelity. There are relatively low fidelity devices which are viewed favorably by 
many trainees and instructors, and some quite sophisticated devices have been maligned 
unbearably by some of the same people. One device used extensively by the U.S. Army 
as an instrument trainer for a number of years was extolled by the device instructors, 
maligned by flight instructors, and described variously as a beast and an aid by trainees! 
A study of the effectiveness of training conducted in the device was less ambiguous: it 
was useless (7). 

. Except to the extent that favorable attitudes increase device use, the effects of 
attitude upon simulator training appear to be practically hiL In a study reported by 
Muckler, et aL {23), negative attitudes tow^d a trainer were induced in an experimental 
poup by stressing the device's low fidelity, while positive attitudes were induced in 
another group by stressing the same device's training effectiveness. During transfer trials 
in the aircraft, both groups were found to have benefited, about equally, from the device 
b-aining, thus indicating that the induced negative attitude did not affect device training 
effectiveness. An interesting aspect of that study was that the negative attitude poup 
required more training in the device to reach criterion, so that the TER value, had that 
measure of effectiveness been used, would have been peater for the positive 
attituJe p-oup. 

No transfer study was found during the current review which indicated that attitude 
per se was a factor influencing simulator effectiveness. On the basis of my own experi- 
ence, it appe^s that just the reverse may be the ^ei simulator tomning effectiveness 
influences attitudes toward simulator Gaining. I have observed abrupt shifts in attitudes, 
particuarly among instructors and Gaining program managers, following demonstrations of 
simulator effectiveness. In one instance, instructors- very negative attitudes toward 
reduced scale paper mockups of a cockpit became favorable when they discovered that, 
unknown to them, their better students were using these "devices" on their own. In a 
study reported by Meyer, et al. (4), pilots' opinions concerning simulator training were 
found to be more favorable following their participation in an effective simulator training 
program than were the opinions of non-participating pilots. 

In spite of a lack of supporting research evidence, there is a consensus among 
trainees, instructors and administrators that favorable attitudes toward simulator training 
increase training effectiveness. This probably is comet in the sense that more extensive 
use will be made of simulators If they are viewed favorably. It may be, however, that 
attitudes are influenced more by simulator toining effectiveness, than the other way 
around, A well conducted ^^test" of ^e training effectiven^s of a simulator may be a 
very influential factor in assuring that its training value will be realized. 

Expectations 

Many aviators accept the proposition that training in a simulator might be helpful 
but view it as less effective than training in an aircraft. It has been my observation that 
simulator training administered under the control of such individuals never exceeds their 
expectations. If simulators are viewed as useful only as procedures trainers or as instru^ 
ment trainers, they tend to be used only m procedures or instrument trainers, even 
though the ^me defaces might be used more effectively by othera who view them as 
offering a peater range of training opportunities. If simulators are viewed as useful only 
for the initial stages of the development of a particular skill, to be follow^ by further 
development of that skill in the aircraft, simulator Gaining is less effective than if they 
are viewed as substitutes for the aircraft to be used for the development of a particular 
skill to criterion before teansferring to the aircraft. While simulator training may not 
always prove as effective as some might expect, expectations appear to place a limit upon 
realized effectiveness by limiting the manner and extent of simidator trmning. 
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Expectations can influence simulator Gaining effectiven^ In more subtle ways as 
well The expectation that a simulator training propam wUl prove ineffective can 
influence its evaluation in the expected direction. Research by Rosenthal (46) has shown 
that even with no intention to do so, an experimenter influences the outcome of his 
reseMch in the direction of his expectations. Since many "tests" of the effectiveness of 
simulator training are conducted by pUots who hold steong views concerning the value of 
simulator vs. aircraft training, we must assume that their expectations ran and sometimes 
do influence the test data. In those instances in which there is real or perceived pressure 
from a higher authority to reach a particular finding concerning the utUity of a particular 
simulator, the effect might be even greater. 

There is an almost infinite number of factors which might shape expectations 
concerning simulator training effectiveness. An obvious factor is prior experience with 
simulator training. The more favorable opinions of pUots toward simulators following 
participation in an effective simulator training propam were noted above. Another factor 
may he their age. Smode. et al. (2), noted that older pilots tend to make poorer flight 
instructors, possibly because of a hesitancy to adopt new teaching methods such as the 
use of simulation. Total flight time is probably also a factor, since the older, more 
experienced pilots are more Ukely to have had unsatisfactory experiences with old 
simulators and typically put peater confidence in In-flight training. 



INCREASING SIMULATOR TRAINING EFFECTIVENESS 

It U unlikely that the effectiveness of any simulator training propam can be 
attributed to a single influence. Instead, all the factors discussed above, and probably 
many more subtle ones, act in combination to produce effective simulator trainmg. Even 
factors which may not be thought influential in Uolation may serve as ratalysts. Effective 
simulator training depends upon a proper combination of hardware, propram, perionnel, 

and other factors, . j • t • i 

Although propess has been made over the four decades since Edwm L,mk 
introduced his first instrument flight trainer, there is still an element of uncertainty 
involved in the design and use of simulators in meeting training objectives. Wheaton and 
Mirabella (46) noted that simulator desipiers have often been more artisans than 
technicians and because of the informal nature of the methods they use, it is difficult to 
reproduce their results or to train others to produce effective devices. The same comment 
can be applied to training prop-am developers, perhaps to an even peater degree. There 
are artisans who devise effective ways of using simulators, even apparently poorly 
designed simulators, but these artisans have not been notably successful in training others 
to produce effective simulator training propams. Conversely, it can be noted that others 
have produced precious little in the way of effectiveness, even though working with 
costly simulators of apparently excellent design. 

While simulator training artisans can sometimes produce spectacular results, there are 
too few such individuals to develop and oontinously update all of the simulator training 
propams required by military and civilian pUot training establishments and operational 
units The present paper was conceived as an attempt to highlight some considerations 
which if attended to, might lead to increases in, simulator training effectiveness. It may 
have that effect in some instances, but I am not convinced that our present data base is 
sufficient to that objective. 

Clearly more research is needed to increase our understanding of factors inlluencmg 
simulator Gaining effectiveness. But a conceptual framework which could make the 
conduct of thnt research more efficient and relevant to generalizable problem solutions is 
lacking Because of this lack, training specialists have no theoretically acceptable design 



models to follow and no effectiveness ^oals to seek, 1 do not see much in tho recent 
SInSdeSnl"'' ^on.omm underpinning for required sUnSoI 

of .i^Zu^""''"^ insuffic^ient information is beinfi disseminated about the design 

of Mmulators and traming pros^ams-both effective and ineffective ones. Most published 
^^n mod""'^;'r'' reports state the identity of the simulator, the experimS 
dts.gn model and the results. Often mformation is not included in the report or 

° SuL'of'u' """"Tf' "-'I' the aUiS^a^ 

om^ne t^vin' ' "TT' f"'^*^" ^^ich should be of interest to 

Ther™necch^7h m^f h ' ''"t''- '° operational training requirements. 

There needs to be much more emphasis upon how the reported results v-re achieved 

are in^^ t h f^^ ^^^'8" simulators and .raining pro-ams 

.mpor an , but pater benefit can be derived from study of existing and 

S^dev Lnmeit T''' T''\T ^"-"^^t- trailing programs 

imuiH f! - The most useful model to follow in the development of effective 
f„,.. °f . P'°^°sd by an existmg application, modified to incornorate 

teatures from other such models as seems appropriate 

».nnr^?^f'™"'f f '"'"^ practitioners and researchers alike need more case study 
eports o Simula or traimnf applications. Such case studies would serve two purposed 
^ln6£t ""n S '° J'" applications; and they woulS pLent 

m^J^fun:-^ ''T f be assembled and studied in efforts to develop conLptual 
models for futu'-i^ applications and to guide research 

effor^^ il, 'I^ ""-f. relegate research to a lesser position of importar.ce in our 
emnh^ i f'h simulator traming effectiveness. I feel that there needs to be more 

so ^t'^'T H ff""""^ ^'''^^'"B training 

he^^T . H f ' ^ conceptual framework can be developed for such research. Our 
^n^TnHo a broader data base^ata which is derivable from present 

^til^^^J - I ^'"'T "^'^^ *° '""'^ the scientific method 

t^^ .^TTr"^'^ '° ' ''^^^ developed. Perhaps the 

r^^arS^L ,h . T^^'f ^ need for better communication among pmctitioners and 
t.earchers about the nature of etfoctive simulator training. Such a step could lead to 

^m'liml^Sort^;:;^.^^^^^^ ^-"'-'^^ 
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